Real-Time Eye Detection and Tracking Under Variable Lighting Conditions and Various Face Orientations

As one of the most salient features of human face, eyes play an important role in interpreting and understanding a person's desires, needs, and emotional states. In particular, the eye-gazes, indicating where a person is looking, can reveal the person's focus of attention. A good eye tracker is a prerequisite of eye gaze monitoring. Robust techniques for eye detection are of particular importance to eye-gaze tracking systems. Information about the eyes can also be used to detect human faces, which will be further analyzed to obtain the face pose information. Many eye tracking methods rely on intrusive techniques such as measuring the electric potential of the skin around the eyes or applying special contact lenses that facilitate eye tracking. This causes serious problems of user acceptance. We have developed a non-intrusive eye tracker that can detect and track a user's eyes in real-time as soon as the face appears in the view of the camera. The eye tracker is aided by the active IR lighting and leaves no markers on the user's face. By combining the conventional appearance based object recognition method (SVM) and object tracking method (mean shift) with Kalman filtering based on active IR illumination, our technique is able to benefit from the strengths of different techniques and overcome their respective limitations. Experimental study shows significant improvement of our technique over the existing techniques.

Publications:

(1) Zhiwei Zhu, Qiang Ji, "Robust Real-time Eye Detection and Tracking Under Variable Lighting Conditions and Various Face Orientations". Computer Vision and Image Understanding, Special Issue on Eye Detection and Tracking, Pages 124-154, Vol.38, No.1, 2005.
(2) Zhiwei Zhu, Qiang Ji, Kikuo Fujimura and Kuang-chih Lee, "Combining Kalman Filtering and Mean Shift for Real Time Eye Tracking Under Active IR Illumination". International Conference on Pattern Recognition, for Oral Presentation, August 11-15, 2002.
(3) Zhiwei Zhu, Kikuo Fujimura and Qiang Ji, "Real-Time Eye Detection and Tracking Under Various Light Conditions". In Proceedings of ACM SIGCHI Symposium on Eye Tracking Research and Applications, 25-27 March 2002, New Orleans, LA, USA.

Demos:

Eye tracking demo 1

Eye tracking demo 2

Multiple-eye tracking demo